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ABSTRACT - / . ' 

This paper first explains the unique role. and 
functions of educational research at a state le.vel .and how it differs' 
from institutional research, and experimental and quasi-experimental 
research. The unique o^po 3: t unities and constraints ate also ' 
elaborated* State educational research has the advantage of central 
and comf)rehensive information as well as a<;cess to very extensive 
information. ' At the same time, the maintenance o$ this capability -is • 
extremely cu-mbersome* As state educational research must t)e able to 
respond to legislative, executive;, and. judicial r equ4.r eaents on 
extremely tight time-lines with Relatively limited resources (for 
example. New Jersey's requirement for thorough 'and efficient (T S E) 
educational processes in the state's schools and districts) the 
system must maintain comprehensive, information on- all components of* 
• education Jihile being able to collect other data, and perform analyses 
without notice. The way in which this decision' oriented system, 
operates is explicfate'd. (Author/RC) 
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USING. PRDDUCT AND PROCESS RESULTS-^ 
FOR. RESEARCH AND PLANNING: 
THE new' JERSEY MODEL . 



Educ.ational.. research at a state level is uniquely 
different from traditional conceptualizations of ^search. 
There is seldom the luxury to conduct ,exp€rinieivt*l''or^Ua si- 



experimental research as^the txim^round' t ime^ggir& gfflpnrg; 
and the generally limited f ocus;..D^--stic'fi .work does not 
satisfy state-wide needs.. It also differs from institutional 
research in that our efforlfs are directly policy oriented, ^ 

"deal with the educational processes and products, and 

• includes greater degree of inferenti'al work. Thougii • 
relying upon a management information 'system, the research 
arm must extend beyond this necessary foundation. 

The state educationa-l research function must embrace 
elements of all these models. It differs primarily in t^hat 

■ it is designed to provide decision makers with'r^pid apd 
comprehensive' descriptive and evaluative data in order to 
provide for effective planning . 'One of the most, frequently 
occuring problems is "that policy decisions and plans are 
made, without a foundation in current and accurate information. 
The research arm must manipulate existing information and/or 
■collect and analyze new information ,to provide needed .data in 
the most useful form for the particular problem being faced. ^ 
Based, on these requirements and'the unique' situation 
of ;«tate level research, a different model 'of research 

. activities and procedures is required., The key elements of 
this situation and the ways in which the model must operate 
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^ in elude, the following: : ^ 



1) Research must be geared to educational policies - 

Most frequently, the research activities ' undertaken fall 
into the categories of modeling the effects of new policy 
directions" or conducting flleld trials of new policies. 
With the financial crises in New Jersey/ we' must closely 
focus our limited resources in tightly defined areas. 
At the same time that the research arm is somewhat 
. restricted in direction, the impact on educational 

operations is more direct than normally found in educational 
research.. 

2) tjuick response times - While tomnorow's problems can 
sometimes be easily predicted, unanticipated introductions 
of education related bills in the legislature is a constant 
fact of life. This ^often requires. -e^Stremely quick research 
efforts to investigate previous findings , analyze 
existing information, ahd report information in a very 
rapid fashion. For example, a^ minimum standards bill . . 
h^s been rapidly introduced and passedi the lower house. 
Consequently, we were asked "to rer evaluate our basic ' 
skills mastery data, review, the policy issues ahd examine 
administrative procedures from other atat^s ^ w^ithin a two- 

' > week time frame. : . 

3) . Comprehensive and Accurate Hana^ement Information -Systems - 

A current and accui;ate management information system is 
absolutely essential to state level research efforts. 
The implicatioi)S af this statement include complete 
' informatiW on over 600 districts and 2600 schools for 
1, 500, 000^ studer^ts . For these components, separate data 



bases must be maintained for finances, school- operations, 
assessment scores, staffing, federal programs, facilities, 
etc. Maintaining data bases on this scope is a demanding 
but necessary task if we are to provide timely information 
to educational decision makers. 

••' • The concept is to try to anticipate any possible' need 

for information and to develop a flexible and compre- 
hensive data base which can be used for all. We will, ^ 

- . jjrobably never |feach the point where we will be totally 

' comprehensive and free of surprises: Unfortunately, we 
were not able to supply immediate informatiqn on the 
transcendental meditation programs in the State when 
legal action. was pursued. 

4) High Benefit Programs .- At the same time that a rapid 
response capability for fast-breaking issues must be 

'.' ^ maintained, we must selectively develop major projects 

which will meet the information needs of a broad variety 
of programs. Our profile and longitudinal study of high 
school seniors, for example, provides information on the 
basic skills issue; secondary and post-secondary articula- 
tion an,d planning data; labor market planning; ' and school, 
district and pr6gram evaluations. 

5) Continued Reprioritization- 4.,At the same time that tight 
■ ' • turn-around times must be maintained, the specific 

elements under study are constantly being re -ordered. 
For example, pressing legislative demands required a 
study of test-based funding., Other On-going ef fort s^ were 
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temporarily- held while extremely intense work was 
conducted over a short time period. The biggest - 
problem in these cases is balancing the requirements 
for maintaining or^-going efforts while focusing on 
the latest set of issues and policy concerns. 
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6) Breadth and Depth - One of the biggest research and 
planning Tftfcc^ation problems faced is the swap-off 
between breadth and depth. Comprehensive and broad 
ranging data bases generally do not have the level of 
detail necessary for program 4.evel decision making. 
For example, our financial files do not go below the 
district level at this time while the only comprehensive 
information on the student level is the statewide test 
^scores. While our Management Information System is 
being over-hauled to remedy these problems, we have 
developed the first of our comprehensive urban data 
bases to overcome our problems of depth. This file 
is built on the sjiudent level an<^ contains over 4300 
cases. Each case contains a complete three year history 
of the student including family. data, socioeconomic data, 
achievement scores, personal attributes, the - student * s 
teachers and their characteristics, building and^ 
p'tij;i^pal char^atiteristics , etc. This highly detail^^^d^ 
infc^rmation is used not only to examine the ef f ecti^ of 
particul>^:.Drograms but also to model projected policy 
changes in -thexducational environment. Our plans are 
to develop similar ti;- depth data bases for the various 
socioeconomic levels ana\community types found in the- 



State, -iihile it is impractical to collect detailed 
information at all levels, by sinking ''wells'* at 
carefully selected points we will be able to make 
^reasonable inferences at all levels. 

7) Research Support for Other Activities - We simply do not 
have the research capabilities to support the" various 
concerns and activities of the entire state educational 
enterprise. Consequently, a strong component of our 
research program is to provide consultative , computer, 

and statistical support to other programs. As is well 

') 

known, a great deal of research work is not the least 
bit glamorous arfd is also very labor intensive. We 
provide^ basic guidance on research design, instrumentation 
sampling, and data collection. The issuing agent then 
procedes to conduct the research. We then provide 
statistical and interpretive work as needed. Examples 

of this type of activity are .found in the evaluation of 

f • - 

Right- to-Read- programs, Title I programs, and evaluat;ions 

of our Educational Improvement Network. This method of 

cooperative activities allows the highest productivity 

with our limited research and statistical expertise 

8) Utilizing external resources in the State - Again, due to''' 
limited resources, we actively solicit and engage other 
educational researchers throughout the State to conduct 
research in areas of our interest. Quite frequently, 
external researchers approach the department for support. 
At the same time that we receive needed assistance, the 
researcher is supported thtjough access to central files 
and broader access to school systems. For example. 



Educational Testing Service research is often consonant 
with our work. Educational researchers at Rutgers also 
.conduct mutually beneficial research projects. 

9) Dissemination and Diffusion - Closely coupled with the 

dissemination and diffusion network for state and federal 
projects, the educational research office also plrovides 
a monthly newsletter to educational decision makers for 
the purpose of keeping them informed of the research 
background and thd current research efforts being 
conducted on topical issues^. An occasional paper 

It 

series is now under develo"pment ^to more thoroughly address 
current issues. This inf ormational^tte^ is absolutely 
essential if we are to accomplish^c^^^ set of 

research goals and avoid unnecessary and costly duplication 
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EDUCATIONAL PROCESSES AND -PRODUCTS 

. The faregoing research and planning model is- one 

that has proven to wo;rk within the opportunities arid 

\ 

constraints 'of one State Department of Education* In 
applying this model to the educational system of New Jersey, 
it is necessary to address both the process and the products 
of the educational system: ^ " 

A. The Process 

New Jersey has a somewhat toique educational situation. 
The stateVonstitution. requires that all children have^ 
a ''thorough and efficient" educatidn. As yoxTlnight 

5 * 
f ' : ' ^ 

expect, this requirement for a quality education was 
. challenged in court. The result was not only a financial 
system over-haul, but also^ a requirement 'for '^t^orough 
and .ef fici€;nt'' (T&E) educational processes in the^state's 
schools'. and districts. The state agency with 'the regional 
and county centers has the responsibility of monitoring 
these focal efforts. \ 

^ The basic process for all schools and districts includes: 

■ ' / a contextual Evaluation of the Icical eduQational 

System with the establishment ^of educational gpals 
through domm'unity involvement ' * . 

* ' / th^ derivation af objectives from tHe$,e goals 

• / the assessment of the Megree of attainment o^ 

• these objectives . ^ 

' / the development ^and*^ implementation of programs- to 
meet the objectives ^ , j' ^ 

/ the. evaluation of the effectiveness of the^e 

pr6gratns^' , j 4 
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This comprehensive program involves an annual plan based ^ 
on a Limited set of criter-ia and a .five-year' comprehensive 
cycle which calls for total review of the local educational • 
.system. - . , ? * * 

In addition to the monitoring, auditing, and approvat 
system now getting underway,- our research efforts must 
be focused* on ■ the effectiveness of this system. ^ Two 
different research foci jnust be distinguished: First, 
the process musr b^ evaluated as a process - both as a - 
tatal entity and for each step individually. The second 
component is addressed to whether the process results in 
improved education or not. On one hand, it is possible 
" to ^perf orm all of the process stepis without, really 
providing an improved education. The" converse may alsd ' 
occur irt that the process (as presented in teports) may 
appear to be woefully inadequate while the actual 
educational prograi!p may be satisfactory. 

It is one of the tasks of educational research" to investi- . 
gate the efficacy of each step of the model internally ^ 
while also relating these/ steps to the products. With the 
over 60*0 districts and 2600 schools^, a very comprehensive i 
and^ complete data file is required. This capability is 
uow'being redeveloped through our T&E efforts and 
through our Managemei:it Information System overhaul. At 
the same time, the extreme' divergence of goal-setting 
.techniques, assessment instruments,' program objective^, 
etc. complicates the process to >an extreme degree. This 
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diversity, however provides an exceptionally, fertile 
ground for comprehei:isive investigations of different 
educational environments. 

B. Products * , , " 

Without any type o\ corifciQii^ touchstone, the use of the 

' ''T&E'' system for statewide research would be an 

\^ 

impossible undertaking. Fortunately, the statewide 

testing program allows a common it>as is for comparisons. 

^ I 

Unfortunately, the tests are r^tricted to performance 
referenced measures of reading and mathematics. The 
tests do, however, report scores for classrooms, schools 
and districts with comparative data for the state, 
county, type of community, and socioeconomic level. ^ 

In adilition to test data, the state has initiated a 
prof ile^ and longitudinal study of high school seniors 
in orde^r to give us' a comprehnsive picture of the 
"products" of the public education system. This 
inforTTiation covers a wide^ vairiety of ^reas related to 
schoQl* programs, school 'success , future plans, -degree 
of preparation, family data, etc. . ' 

COMBINII^G THE ELEMENTS 
A research ptc3gram with extensive and accurate data' . 
coupled with process' and pfoduct evaluations provides a 
comprehensive and complete picture of our educati\Onal 
environment. With, the assessment data on every school, 
and distri^ct as well as the school descriptor and 
program data, we can partition the information in an 



infinite variety of ' ways to address specif ic -ajroblems . 
For example-i we wqre 'faced with a rapid need to conduct 
an evaluation of the Right- tp-rea^program. , It was a 
straight-forward tasks to identify the program schools, 
^,the time of implementation, the intensity of tiie '^program 
and parttjtion thg schools into ..suitable sec i^o economic ^ 
an'd communitV groupings. From this point, a control sarrple 
of schools was drawn and comparisons made using the 
st-atewid.e test scores. A study of the effectiveness and 
concentration of federal projects was conducted in a 
similar straight-forward manner. 

Unfortunately, reality is never as neat and/'conplete as. 
tine model. All applications do not have such parsimonious 
conclusions. The T&E c<3mDonent is still in field test 
a^id a well operating^sys tem also nasks the grindirg l?.bor 
and frustratior; of building and maintainine comprehensive 
dat^ basest 

' It. would be incorrect to inoly that the above described 
data oriented system can provide answers* to all state^ 
level problems and concerns There are problenis and 
prbcesses that have to be addressed which require other . 
means and methods: 

1) Non-quantifiable data - Obviouslyf paper-and-penci 1 
measures of achi^ement and survey r^sulrs will teli 
us little of the affectiVe learning which ^res 
in the school Likewise, critical eler.ents s-:h as 
school climate will not show uo in the data 
Consequently, field and case work is^jse- :: ree: :"e?v 
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graduate .s tudent .intern program is used here to do 
rapid searches of boLu primary and secondary 
documents while simultaneously develop,ing a. compre- 
hensive and recent profile of the New Jersey situation. 

' — fc 

€CMCLUSIOr> ^ ^ 

The educational research system, the process and 
product rec'p' rc-.cn ts of the "thorough and efficient'* system, 
and the .special efforts designed to fill gaps provides a 
cc^prehensive and flexible s^3ten fpr investigating educational 
concerns m the State *^ 

The basic skills issue, for ex amp le , is a case 
vhere the syster. v:as" brought to bear on short time lines 
zo provide policy information The research capability 
-•v-iQ jsed to reviev' pre%^ious findings, analyze testing data 

co-^urirv t -ne .nd bv socioeconomic a^dtus, identify ' * 
r:'aze levels and locations of str^erqths and weaknesses, t 
I '.isaze or^zrrars and orocesses, and correlate educational 
. - --educational variables with success. 

The capability is now being employe^ to mqdel the 
effects of ^he sta;e compensatory education bill. Areas and 
^.ver .c-^^noer exoloration include riethods of determining 
:^-o:n: level, ^-^.-c.---- ^ c*--,^ -ion . orog;ram ^ " 

cT'^a--'- arj success of ororra^ns 

T"^ e air o : the s rate educational research arm is 
t- :^r:'. :;e ti~el".' accurate, and easilv conorehensib le data 
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to educational decision makers. Traditional problems 
have been in collecting and maintaining this data capability 
The procedures we have developed and implemented have 
shown themselves to be successful in meeting .these unique 
research and planning demands. 



